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Executive Summary 


This report summarizes key activities and contributions and offers recommendations for 
project operations. The pilot project was very successful in building a solid foundation for the 
Early Childhood Mental Health (ECMH) Consultation Program. Practitioners and other 
regional stakeholders clearly indicated that the project is providing a much needed and valued 
service. At a specific level, ECMH has met the needs of many young children, families, and 
practitioners through consultation, referrals, and sharing of resources. At a broader level ECMH 
has increased the awareness and access across regions through the development of resource libraries, 
provision of networking and educational opportunities, and participation in or facilitation of 
systems building efforts. 

The pilot began by examining needs, gaps, and issues in the Northwest, South Central, and 
Southwest Regional Keys. Based on this information staff outlined criteria and procedures for 
service delivery and developed strategies to document consultation and outreach activities and 
issues. Over the course of the two year pilot, the seven member team served 226 children and 133 
early learning programs. Programs most frequently sought help for children experiencing 
difficulties with self-regulation, aggression/ acting out, and attachment or interaction issues. These 
findings were similar across Regional Keys and urban, suburban, and rural settings. As expected, 
early care and education programs at the STAR 1 level requested services most frequently. A 
surprising finding however was that STAR 4 programs that reached this designation through 
NAEYC accreditation requested consultation services at much higher rates than STAR 4 
programs that reached this designation through Pennsylvania’s Keystone STARS Performance 
Standards alone. 

ECMH consultants referred 58% of their cases to other support services. Referrals were 
most frequently made to early intervention, however referrals were also made to STARS Technical 
Assistance (STARS TA), children’s mental health agencies, and other supports such as 
pediatricians, child evaluation units, audiologists, and adult mental health agencies. On “average” 
ECMH consultation lasted for just under three months and was provided most often for older 
toddlers. Consultation ended for most cases (66%) because children met identified behavioral 
goals or because children or programs were referred to other services. It appears that consultants 
needed to provide services for longer periods of time when children had less well-developed social 
emotional skills/more challenging behaviors as reflected in their developmental screening scores. This 
relationship was statistically significant (p<.01). By the end of the pilot, 200 children were 
discharged from consultation services. 

Many early care and education practitioners gave high praise for the program. They clearly 
valued working with ECMH consultants and saw on-site assistance as the most valued and helpful 
aspect of the program. Positive changes were noted in the perceptions, behaviors, and skills of staff 
as well as positive, though gradual, changes in the children’s behaviors. Practitioners felt these 
changes were a direct result of their interactions with ECMH consultants. Survey findings 
indicated that while consultation facilitated the development of new skills, it primarily helped 


1 



practitioners meet the social emotional needs of children for whom support was requested. The 
following comment reflects this enthusiasm: 

“It has been the most valuable resource I have experienced in my 10 
years directing this center. ” 


Regional stakeholders felt that the project made valuable contributions to the State’s infant 
toddler mental health system over the course of the two year pilot. The project increased awareness 
of children’s mental health issues; increased knowledge of services, supports, and child development, 
AND increased access to educational materials, referrals, support services, training, and collaboration 
between child serving systems. The voices of stakeholders and the experiences of consultants clearly 
reflect that there is much work to be done. Many families, particularly in rural areas, still need 
information, education, and support in mental health issues and services and early care and education 
practitioners and administrators continue to need more resources, training, and networking 
opportunities to better address the mental health issues of young children and their families. 

The pilot brought to light the importance of building an identity and developing 
procedures to align with the program’s mission and identity. Staff found that the program’s 
identity provided a framework to standardize an overall approach (who we serve, how we serve, the 
scope of services, etc.) and guide the development of program procedures. They also found that 
this “framework” needed to be flexible enough to allow consultants to individualize some 
procedures to meet specific needs within their Regional Keys. As the project moves out of the 
pilot phase, refinement of the program model is critical. The evaluators recommend that the 
project develop a detailed plan to identify goals, core program activities, and program outcomes. 
This plan should include establishing clear benchmarks for the implementation of program 
activities and the measurement of program outcomes. Measurement of program activities will 
help ensure fidelity to the model and highlight needed refinements and measurement of outcomes 
will establish program impacts. 
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The Early Childhood Mental Health Consultation Program 

Contributions of the Pilot Project 


“Looking is not seeing. Listening is not hearing. It is possible to miss so much that is 
right in front of us if we lack the categories and skills to notice. The greatest of these 
skills is, perhaps, to put aside our expectations, and to stay open to the actual. ” 


The Early Childhood Mental Health (ECMH) Consultation Program listened to, informed, 
and supported the early childhood community through consultation, outreach, and systems building 
activities. Consultation focused on helping practitioners promote healthy social emotional 
development and provide support to young children and their families. Outreach activities were 
directed at increasing awareness of and access to area resources and systems building activities focused 
on encouraging collaboration and linkages between child serving entities across the State. The 
program’s efforts have been broad and have made a difference in Pennsylvania. 

This report provides an analysis of the pilot project’s key contributions and impact, 
summarizes key lessons learned, and offers recommendations for continuing operations. The 
report’s primary focus is on activities and initiatives conducted in the program’s three pilot sites. 
These sites were located within the Northwest, South Central, and Southwest Regional Keys. 
Positioning the pilot within the Regional Keys was instrumental to the project’s successful entree 
into early care and education programs. The ECMH Consultation Program was formerly known 
as the Infant Toddler Mental Health (ITMH) Project. It is referred to as ECMH or “the project” 
throughout the report. 


IDENTIFYING EARLY NEEDS AND ISSUES 

ECMH staff and evaluators conducted surveys with 
early learning practitioners and regional stakeholders at the 
beginning of the pilot. Practitioner survey findings, which 
are highlighted to the right, revealed substantial child and 
staff-related issues. Similarly, regional stakeholders 
expressed numerous concerns when asked about gaps, 
issues, and needs within systems serving infants, toddlers 
and caregivers. Their concerns included inadequate funding; 
restrictive eligibility criteria; limited awareness, knowledge, 
and access; limited family-centered services; and a lack of 
comprehensive and coordinated services. Stakeholders felt 
these issues left young children’s mental health needs under- 


Practitioner Survey Finding 

37-66% children in programs 
displayed atypical behaviors, 

33-36% of children had been 
expelled [from infant / toddler 
programs], 

25-39% of staff left programs 
because of children’s 
behaviors, and 

Up to 89% of practitioners 
requested some type of 
support around infant and 
toddler mental health issues. 


1 Donald M. Berwick, from the foreword of Organizing for Quality: The Improvement Journeys of Leading Hospitals in Europe and the United 
States Taken from Rand Corporation Research Brief: Organizing for Quality Inside the “Black Box” of Health Care Improvement in Europe and 
the United States http://www.radclifie-oxford.com/books /bookdetail.aspx?ISBN= 1846191513 
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identified and misunderstood AND resulted in an under-utilization of existing services and a 
shortage of comprehensive, coordinated services. Three themes emerged from early surveys. 

1. A need to raise awareness about young children’s social emotional needs, 
challenges, and issues 

2. A need to maximize access to early childhood mental health resources and 
services for children and caregivers 

3. A need to provide flexible access to and funding of comprehensive and coordinated 
services. (See Appendix A for a complete list of early survey findings). 

Issues, concerns, and early findings clearly 
pointed to the need for a multi-faceted service 
deliveiy model that could provide on-site 
consultation, use flexible eligibility criteria, 
promote the importance of relationships, 
maximize access to mental health, resources 
and services, and enable caregivers to understand 
and support children’s social emotional 
development. 


Figurel. Service Delivery 



Use flexible 
eligibility criteria 


Maximize access to mental 
health resources & services 


Enable caregivers to 
understand & support 
children’s social emotional 
development 


4 






DEVELOPING A SERVICE DELIVERY MODEL 


The pilot began by examining needs, gaps, and issues of early care and education 
practitioners and other stakeholders in the Northwest, South Central, and Southwest Regional 
Keys. This involved formal and informal discussions, interviews and surveys with regional 
stakeholders and the reviewing of relevant literature. Based on this information staff outlined 
criteria and procedures for service delivery and developed strategies to document consultation and 
outreach activities and issues. Careful documentation, feedback, and discussions allowed the 
ECMH team to identify effective and ineffective strategies, desirable refinements, and needed 
developments. Collaboration around these activities helped the program refine THEIR philosophy 
and approach to outreach, consultation, and referral; led to the development of standardized forms 
and procedures. 

What is the Project’s Approach to Consultation? 

ECMH consultation focused on helping practitioners learn to effectively address the 
behavioral needs and challenges of their children. Consultants became resources to practitioners 
supporting them as they learned to address the complex issues of early childhood mental health. 
They did this by helping practitioners observe children’s needs and create settings that nurtured 
healthy development, communicate child-related concerns to families, develop plans that address 
specific [children’s] behavioral concerns, and access additional supports such as early intervention 
and mental health agencies. Furthermore, consultants partnered with families around children’s 
needs. ECMH consultation, 

1. Focuses on helping children remain and succeed in early learning settings, 

2. Is child specific and is not restricted by eligibility criteria 

3. Encourages and supports family involvement, 

4. Brings services and resources to practitioners in their settings, 

5. Provides services directly to practitioners on behalf of children, so that 
practitioners develop the knowledge, understanding, and skills necessaiy to 
work with children on an ongoing basis, 

6. Is intended to be short-term, and 

7. Refers children, families, and/or programs to specialized supports (such as 
mental health agencies, Early Intervention, and STARS Technical 
Assistance) when they are needed. 

What is the Project’s Approach to Outreach and Systems Building? 

Defining outreach and systems building went hand in hand with defining consultation. The 
program’s approach to outreach and systems building was multi-dimensional. It involved 
assessment, marketing, resource development and coordination, education, and systems support / 
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systems building. The first critical component was gathering, coordinating, and disseminating 
information. The second was helping to ensure that existing systems were being used, that services 
were being offered in integrated fashions, and that entities such as local interagency coordinating 
councils, CASSP advisory groups, and community engagement groups, had opportunities to 
collaborate. The third was supporting systems change by trouble shooting issues and keeping key 
agenda items “on the table” through meetings, materials, and other communications. Individualizing 
regional approaches proved to be an important aspect of effective outreach and relationship building, 
especially early in the pilot. Unlike consultation, the approach to outreach and systems building 
needed to be less standardized because of regional differences in geography, resources, and needs. 

The team recognized early in the pilot the importance of developing a central or unified 
identity. Questions such as, “what services would be provided”, “to whom services would be 
provided”, and “how services would be provided” were subjects of many discussions. Shared 
approaches and experiences and a focus on standardizing ECMH policies and procedures guided 
the development of this identity. The team also recognized the importance of incorporating 
flexibility in its approaches and procedures. This flexibility enabled staff to address unique regional 
needs such as rural versus urban settings and new versus established contacts. The development of 
a service delivery model that can be replicated across the State has been one of the most 
important contributions of the pilot project. As the project moves forward, it will be important 
to refine the core elements and make sure that fidelity to these elements is maintained. It will be 
equally as important to allow enough flexibility to support the unique needs and demands of each 
Regional Key. 
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BUILDING CAPACITY IN REGIONAL PROGRAMS 


“\Lasting change] involves embedding early childhood mental health principles and 
staff in all relevant state offices to ensure a culture to support early childhood mental 
health in all young children .” 

Excerpt from an ECMH staff interview; interview notes are located in Appendix A. 

Building capacity among early learning providers and other partners in the region was a 
primary focus of the program’s work. Consultation, outreach, and systems building provided the 
vehicles to build capacity, but thoughts like the above sentiment guided the team’s spirit and 
commitment. Over the course of the two year pilot, the seven member team served 226 children, 
133 early learning programs and partnered with many agencies, departments, and state offices. 

Consultation Services 

Consultation was requested for 226 children, from 133 different early learning programs. 
Consultants provided practitioners with strategies, advice, information, and support in three key 
areas. These areas were (1) addressing child and programmatic issues, (2) accessing specialized 
services and supports, and (3) promoting practices that nurture young children’s social emotional 
development. Practitioners requested assistance most frequently for concerns around 
attachment/withdrawal, self-regulation, and aggression/ acting out with services occurring most 
often in urban and suburban centers. Programs were somewhat different in their levels of program 
quality, but fairly similar in levels of staff education, and training. Most programs had either no 
staff (31%) or one staff (50%) with an Associates degree or higher and one to two staff members 
with recent professional development training on infants and toddlers (71%). A more complete 
summary of child and program characteristics is provided in Appendix C. However, a snapshot of 
“typical” ECMH case characteristics is provided below. 


Table 1. Most Typical Characteristics of ECMH Cases 


Case Characteristic 

Case Statistic 

Based on 
information from 

Reason for requesting consultation 

Attachment/interaction issues (25%) 
Self Regulation issues (29%) 

Acting Out/Aggression (29%) 

226 of 226 cases 

Average age of a child at the 
beginning of consultation 

28 months 

(Range: 6 weeks to 5% years) 

216 of 226 cases 

Early learning facility type and 
setting 

ECE center in urban setting (36%) 
ECE center in suburban setting (47%) 

117 center-based 
programs 

Early learning centers’ STARS 
level 

Average length of case 
consultation 

STAR 1 (40%) 

Starting with STARS, STAR 2, STAR 3, 
and STAR 4/accreditation (13-16%) 

2 months and 27 days 
(Range: 1 to 305 days) 

196 out of 226 
cases 

192 of 200 
discharged cases 

Reason for discharge from 
consultation 

Children met identified goals (55%) 
Referred to another service (20%) 

192 of 200 
discharged cases 
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An early concern of some regional 
stakeholders was that ECMH consultants 
would duplicate the work of existing area 
programs. However [to the contrary], 
referring children, families, and programs to 
needed services and providing education 
about existing regional services/supports was 
and continues to be an important aspect of 
the mission. ECMH consultants referred 
58% of their cases (children/programs) for 
evaluation or assistance from additional 
support services. Additional referrals were 
most often made to early intervention (El). 
Referrals were also made to STARS 
Technical Assistance (STARS TA), 


MD 

ECE 



Figure 2. ECMH Consultants’ Referrals 


children’s mental health agencies, and other 
supports such as physicians and child 
development units. In addition, a small 
number of cases were referred to multiple 

services, e g., El and STARS TA; El and a physician; and El and adult mental health. 

Outreach and Systems Building Activities 

The team conducted numerous outreach activities and successfully engaged stakeholders in 
early care and education, early intervention, behavioral health, higher education, and health and 
human services. Stakeholders included practitioners, parents, coordinators, therapists, 
administrators, pediatricians, state officials, and educators. The vast majority of the outreach 
activities involved increasing awareness of ECMH and building linkages within regions and across 
the state. Staff used outreach efforts to establish the project’s reputation as a source of education, 
consultation, and support to early learning facilities and agencies. 

The team engaged in a variety of communication strategies designed to build relationships, 
increase awareness about ECMH services, coordinate and promote comprehensive services, and 
address gaps, needs, and issues within early childhood mental health. State level systems building 
activities included the Infant Toddler Mental Health Symposium, professional networking events 
and conferences, and the formation of committees and workgroups. The ECMH team provided 
over 120 trainings, professional development opportunities, and presentations; developed regional 
resource libraries and disseminated materials; assessed regional needs, perceptions, and resources; 
and identified and developed curricula. A few of the topics addressed in trainings and presentations 
included social emotional development, behavior management, responsive caregiving, consultation 


8 


and mentoring, and accessing support services. A snapshot of these activities is provided in Table 
2; a complete list is located in Appendix B. 


Table 2. Examples of ECMH Outreach Activities 


ECMH Staff 
Activities 

Examples 

Engaged in 372 
meetings, calls, 
mailings, etc. 

Contacts and partners included Infant and Early Childhood Mental Health Coalition, Chatham 
University, PA Department of Public Welfare, Early Intervention, Early Head Start, local pediatricians, 
Zero to Three, Local Interagency Coordinating Councils, and Capital Area Early Childhood Training 
Institute,. 

Provided 68 

trainings/ 

professional 

development 

opportunities 

Topics included observation in infant/toddler classrooms, helping children manage anger, looking 
through different eyes at challenging behaviors, supporting children’s social and emotional 
development, and building relationships with infants and toddlers. 

Made 61 
presentations 

Audiences included Beginning with Books, Starting Early Together, Project Early Childhood Higher 
Education Opportunity, United for Children, PA Child Care Association, and STARS Technical 
Assistance and Mind in the Making. 

Engaged in 53 
resource 
development 
activities 

Developed regional resource libraries, created educational materials, identified/reviewed curricula and 
assessment tools, compiled targeted resources for stakeholders, acquired community resource guides 
and developed county resource lists, and installed “AutoLibrarian” software. 
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IS THE PROJECT MAKING A DIFFERENCE? 


The Impact of Consultation 

ECMH consultation helped many children receive needed supports both within and 
outside of the classroom. By the end of the pilot, most children (68%) were discharged from 
consultative services because their issues had ceased, had significantly decreased (i.e., they met their 
behavioral goals), or because they had been referred to other services. 2 In addition, consultants 
assisted a number of programs to obtain guidance and support around quality improvement issues 
through referrals to STARS TA. Consultation services ended for, 


74 children (38%) because they met 
behavioral goals, 

57 cases/children or programs (30%) 
because they were referred to another 
service(s) 

28 children (15%) because of parental 
choice (e.g., chose to remove their child 
from the referring program, pursue 
other services), 

13 families (7%) moved, and 
20 children (10%) because of provider 
choice (e.g., expulsion or because the 
provider chose to discontinue 
consultation services - further 
explanations not provided.) 


Figure 3. Reasons for Discharge 
from Consultation 



Consultants supported practitioners for longer periods of time when children had less well- 
developed social emotional s kill s and/or more challenging behaviors. The finding is based on a very 
moderate relationship (r=.237, p<.01) between children’s ASQj. SE scores and the length of 
consultation services. 

Early learning practitioners offered very high praise of the program’s services. They felt 
that interactions with ECMH consultants not only improved the perceptions, behaviors, and skills 
of early learning staff, but more importantly, helped staff meet the needs of children for whom 
support was requested AND meet the needs of other children enrolled in their programs. This is 
reflective of how the program is staying true to its mission of helping early learning facilities 
address the social emotional needs of children with concerning behaviors and as well as providing 


2 


Eight children (4%) were excluded from the analysis due to missing information. 
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practitioners with support, information, and strategies that support early social emotional 
development more broadly. In the words of two practitioners, 

“I would recommend this program to anyone. If it was not for [the ECMH 
consultant ], this child would not be allowed to stay enrolled in our program. ” 

“It has been the most valuable resource I have experienced in my 10 years 
directing this center. ” 


Table 3 . Impact of ECMH Consultants on Practitioners’ Skills* 


Skill 

No Change 

Somewhat or 
Greatly Improved 

Already 
Proficient/No 
Change Needed 

Understanding child’s history and current behavior 

14% 

71% 

14% 

Understanding family situation 

14% 

64% 

21% 

Making referrals for child 

6% 

73% 

21% 

Meeting socio-emotional needs of this child 

6% 

80% 

14% 

Meeting socio-emotional needs of other children 

14% 

79% 

17% 


*Based on responses from 50 practitioners 


Consultation services appear to have been well -received and valued across the three regions. 
However, these findings represent feedback for only 25% (n=50) of all discharged cases in part 
because feedback surveys were developed later in the pilot (See Appendix D for a detailed report of 
findings). As the project moves forward, it will be important for consultants to send out feedback 
forms in a timely fashion and for the project to track the return of these forms. 

The Impact of Outreach and Systems Building Activities 

ECMH staff promoted education and collaboration across systems and across disciplines 
through their work in outreach and systems building. These impacts are a result of undertakings 
such as the Infant Toddler Mental Health Symposium, the development and coordination of 
regional resource libraries and participation in meetings, trainings, and presentations. Regional 
stakeholders felt the ECMH Program made valuable contributions to the State’s infant toddler 
mental health system over the course of the two year pilot. 3 These contributions fell into one of 
three categories - increased awareness, increased knowledge/ understanding, and increased access. 
ECMH has changed the statewide early childhood mental health system by increasing awareness of 
children’s mental health issues; increasing knowledge of services, supports, and child development, 
AND increasing access to educational materials, referrals, support services (e.g., El, MH, medical 
services), training/professional development, and collaboration between child serving agencies. 3 4 


3 Follow-up surveys were conducted with stakeholders at the end of the pilot. Conclusions are based on survey responses from 85 
stakeholders in the Northwest, South Central, and Southwest Regional Keys. 

4 Themes are based on results representing the greatest percentage of stakeholders’ responses to survey questions. 
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Important working relationships have been 
formed with regional and statewide partners. 

These partners include Chatham University, 

Capital Area Early Childhood Training Institute, 

Office of Child Development and Early Learning 
(OCDEL); Office of Mental Health and Substance 
Abuse Services (OMHSAS); Office of Children, Youth, 
and Families (OCYF); and Office of Medical Assistance 
Programs (OMAP).These collaborations brought and 
continue to bring interested parties “to the table” to share 
information and work together on solutions. One prime 
example of these efforts was the recent training of 
hundreds of early care and education directors on the 
Ages and Stages Questionnaires: Social Emotional 
[screener]. This training focused on promoting the 
importance of screening and greater understanding of 
referral processes. Another key example of these efforts 
was the sponsoring of 30 professionals in Chatham 
University’s Infant Mental Health Certificate Program. 
Such collaborations are pivotal in enhancing education 
and mental health services at local, regional, and state levels. 


Impact across Regional Keys 

Increased awareness, knowledge, 
and understanding of, 

• infant toddler mental health issues, 

• typical and atypical child 
development, 

• available services and supports. 

Increased access to 
educational materials, 
mental health services 
medical services, 
referrals 

opportunities for collaboration. 
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PLANNING AHEAD 


Lessons Learned 

The pilot brought to light the importance of building a core identity. Staff found that 
developing a “project identity” provided a framework to standardize their approach (who we serve, 
how we serve, the scope of services, etc.) and guide the development of policies and procedures. 
The needs of practitioners and other child serving systems were realized and met through the 
implementation and ongoing re-evaluation of project policies and procedures. The team also 
found that the “framework” needed to be flexible enough to allow consultants to individualize 
some strategies within their Regional Keys. Ongoing internal review and support helped the team 
forge their identity by keeping them “on the same page” in terms of philosophy, goals, and 
procedures. The importance of periodically revisiting procedures and issues cannot be overstated as 
this was essential for a unified approach to ECMH service delivery. 

As part of the evaluation and as part of project procedures, the team developed a detailed 
data collection plan and worked hard to maintain project records. This task sometimes proved 
challenging in terms of staff time and/or s kill s and sometimes resulted in missing, delayed, and/or 
incorrect information. Uniform, accurate, and timely data collection is essential to project 
operations. Information about consultation and outreach activities (the process data) is needed to 
document what the program does and what the program needs. Outcome data, such as 
practitioner feedback surveys, is needed to document what differences project services make for 
young children and their caregivers. Taken together, these data can make the case for systems 
support and systems change - both important contributions for children, families, programs, and 
agencies across the State. 

The data 5 suggests that the ECMH Consultation 
Program is making a difference. However, the voices 
of stakeholders and the experiences of consultants 
clearly reflect that there is much work to be done. 

Many families, particularly in rural areas, still need information, 
education, and support in mental health issues and services. 

Consultation notes suggest that families still refuse available 
supports due to fears, concerns, and/or misconceptions about 
mental health labels and services. And early care and education 
practitioners and administrators continue to need more resources, 
training, and networking opportunities to better address the 
mental health issues of young children and their families. 

When asked, “what would you change about the ITMH Project,” the most common 
suggestions of stakeholders suggested increasing the projects’ marketing efforts to “get the word 


Stakeholders’ Needs 
2008 Survey Findings 

67% needed more educational 
materials for themselves and 
their families 

62% needed more 
opportunities to network with 
practitioners and other 
professionals and 

54% wanted more training on 
early childhood mental health. 


5 Data such as the number of children served, the number of children discharged because goals were met or additional services 
were identified, and feedback from practitioners. 
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out,” increasing the number of ECMH staff, expanding the age of children served to include 
preschoolers, and offering training for parents. Last but not least, ECMH staff offered 
suggestions when asked what they saw in the project’s future. Some of their suggestions are 
provided below. 

★ Consultations services to young children through age 5 years, 

★ Additional training for early care and education staff on mental health and 
family issues, 

★ Expanded education for legislators on the importance of social emotional 
development in young children, 

★ Directed outreach and marketing of the project to families especially in rural 
counties, 

★ A continuing education system and professional development opportunities for 
early care and education staff focusing on social emotional development, and 

★ Support for ECMH staff that includes regularly scheduled reflective 
supervision, and expansion of consultants’ role to include systems building. 


Conclusions and Next Steps 

The pilot enabled the team to lay a solid foundation for services that are making a 
difference across the State. Practitioners and other regional stakeholders clearly indicated that the 
project is providing a much needed and valued service. At a specific level, ECMH has met the 
needs of many young children, families, and practitioners through consultation, referrals, and 
sharing of resources. At a broader level ECMH has increased the awareness and access across 
regions through the development of resource libraries, provision of networking and educational 
opportunities, and participation in or facilitation of systems building efforts. 

As the project moves out of the pilot phase, refinement of the program model is critical. 
The evaluators recommend that the project develop a detailed plan to identify goals, core 
program activities, and program outcomes. This plan should include establishing clear 
benchmarks for the implementation of program activities and the measurement of program 
outcomes. Measurement of program activities will help ensure fidelity to the model and highlight 
needed refinements and measurement of outcomes will establish program impacts. Care should be 
taken to closely review rating categories of key variables (for example “Reason for Discharge”) to 
make sure needed information is being captured. 

As the project moves to a web-based data collection system, the evaluators strongly 
encourage the program to provide ongoing, face-to-face training and technical support to staff to 
help ensure data integrity. Training should include data entry and editing procedures; expectations, 
guidelines, and tips for managing electronic case files; and strategies for identifying and correcting 
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data entry errors. It will be important for the program to consider how the new database can 
support reporting needs and may be helpful to develop a data report matrix [table] to outline these 
needs. Information to include in the matrix is report type, report audience, purpose, required 
information, and reporting frequency. 


While numbers show that change is happening, the voices of families and providers show 
why the changes are important. Selective databasing of family, practitioners, and agencies’ quotes 
and stories is suggested. This type of information can be a wonderful complement to the project’s 
quantitative data. In addition, the evaluators recommend refining the documentation and 
measurement of outreach and systems building activities [and outcomes]. The team may wish to 
investigate strategies such as community asset mapping or systems change assessment tools. 
Evaluation of the project’s activities and outcomes is critically needed to ensure fidelity of the 
model and clarity of project impacts. 


Evaluation Questions to Consider in the Future 


1 . What do your indicators and outcomes say about the 
project’s performance 

2. How much change is indicative of success? 

3. What other factors — outside of the project - influence 
identified outcomes? 

4. What are the implications of your results for future 
planning and implementation? 
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Appendix A 

Summary of 2006/2007 Surveys and Interviews* 


Table 4. Summary of 2006/2007 Surveys and Interviews 


Report 


Major Findings 


ECE Directors’ 
Surveys conducted 
by ECMH staff with 
ECE practitioners 
in Western (n=248) 
and South Central 
(n=90) 


Results of the surveys by percentage of practitioners: 

With children displaying atypical behaviors — W=37%, SC=66% 
With children who have IFSP— W=25%, SC=40% 

With staff who left due to children’s behavior — W=25%, SC=39% 
Who dismissed children due to behavior — W=36%, SC=33% 


Summer 2006 


Who approached parents — W=73%, SC=79% 

Who are comfortable offering parents guidance — W=86%, SC=78% 


Requesting supports — 

Training— W=33%, SC=72%, 
Resource guides — W=33%,SC=89%; 
TA — W=17%, SC=51 %; 

Website— SC=56%, 

Support group — W=12% 


A Qualitative 
Analysis of 
Stakeholders’ 

Views of the 
Pennsylvania Infant 
Toddler Mental 
Health Project 


Who need more IMH knowledge — W=82%, SC=74% 


The need to maximize access to ITMH resources and services — increased recognition will lead to increased seeking 
and utilization, a barrier to this is the narrow focus of ECE and MH/H practitioners 

The need for flexible access to comprehensive services — re-conceptualize ITMH to be more holistic and family- 
oriented as well as increase knowledge of ECE and MH/H practitioners on how to access services and supports and 
when to do so 


42 stakeholders 
(MH, H, El, and 
ECE)— from NW, 
SW, and SC. 


What ECMHC can do — lead community- and region-wide trainings and advocacy activities; disseminate info on typical 
development, MH issues, available resources, and importance of quality, socio-emotional environment; serve as liaison 
for ECE, El, and MH practitioners; and provide info and support to families and ECE practitioners 


Fall 2006 


CASSP Survey 


56% worked with children 0-3 years and 71 % worked with children 3-5 years 


87 participants of 
the 2007 Children's 
Interagency 
Training 

Conference of the 
CASSP Institute 


Services provided by respondents included interventions in home or community (82%) followed by assessment/referral 
and service coordination both at 72%, interventions classroom (56%), and training and technical assistance (53%) 

74% of respondents identified the need for ECMH services as mostly or always prevalent in their target population 

Barriers to training — lack of knowledge about the need for ITMH (89%), lack of knowledge about child development 
and typical behavior (80%), lack of trained professionals (78%), and other reasons cited by 18% of respondents 
included money for trainers and trainees, lack of available trainings on specific topics, and few competent trainers and 
evidence based curriculums. 


August and 
September 2007 


Access barriers to services — need for diagnosis to receive services (89%), lack of knowledge of services (88%), lack of 
transportation to services (71%), stigma related to receiving mental health services (69%), cost of services or lack of 
insurance coverage (67%), and other reasons cited by 18% of respondents were more systems barriers such as lack 
of coordination among systems and professionals serving the children, lack of available services, and waiting lists 

Systems barriers to services — lack of coordination among different systems serving infants/toddlers and their families 
(80%), high turn-over among TSS workers (75%), bureaucracy in getting services (69%), and long waiting lists for 
services (68%). 

94% saw a need for increased professional development specific to ECMH 

Areas of interest included: autism, effects of trauma on child development, infant mental health, socio-emotional 
development and attachment, behavioral health, and various therapies including art, play, and dance. 

Consultation (52%), and play therapy/recreational therapy/social skills (50%) 

54% would be interested in offering professional development training in their area 

95% would like ECMHC to contact them for more information 
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Table 4. Summary of 2006/2007 Surveys and Interviews (cont’d.) 


Report 


Major Findings 


CASSP Survey 
(cont’d.) 


Trainings should be offered on-line, pay participants to attend, and center around topics such as communication, 
collaboration, and service delivery across agencies. Specific areas of training identified were Applied Behavior 
Analysis, assessments, accessing services, bridging from El to AIU, child development, and PATHS. 


Cross trainings should bring together the following groups of professionals: MH & ECE; MH & Education; El & ECE; El 
& behavior therapists; and El, MH & Pediatricians. 


Specialized areas of training respondents currently have include — behavior management (63%), assessment (60%), 
program consultation (52%), and play therapy/recreational therapy/social skills (50%) 


54% would be interested in offering professional development training in their area 
95% would like ITMHP to contact them for more information 


ECMH Staff 
Interview and 
Surveys 


The ECMH project has changed the statewide ECMH system by increasing levels of discussion and awareness, 
training ECE staff in ECMH, building relationships across systems (OCDEL, OMHSAS, OCYF, and OMAP) serving 
infants/toddlers, increasing efficiency of referral process and intervention, and reducing the stigma related to ECMH. 


(Project Manager 
interview and web- 
based staff survey) 


Barriers that hinder changes in ECMH system include lack of funds at state level for prevention and intervention, failure to 
qualify for services when only social emotional issues are present, limited access to services especially in rural counties, 
less developed systems to support ECMH in areas outside Pittsburgh and Philadelphia, and stigma related to ECMH. 


September and 
October 2007 


During the relationship building period, concerns and resistance developed among El, some ECE facilities and other 
agencies that provide outpatient services. These concerns focused on confusion about the ECMH project's roles and 
misconceptions of ECMH as “taking over” the other agencies’ roles. Though there are varying levels of acceptance of 
these changes in the ECMH system especially among El, some ECE facilities, and agencies providing outpatient 
services. However, Mental Health has been very supportive and collaborative at both the state and regional levels. 


Suggestions to increase acceptance include meeting individually with specific agencies and ECE facilities around 
individual children; marketing ECMH system to gatherings of ECE practitioners; educating El on how ECMH will 
compliment their work, address an unmet need, and increase El referrals; and providing highly skilled parent/child 
therapy trainers in key areas around the state where there are low levels of acceptance. 


Lessons learned so far: it is hard to build systems — one must proceed with caution and continually reinforce 
relationships that have been forged, time spent prior to the first visit sending forms and building relationships and after 
a referral following up with a call — is time well spent, must involve parents in meaningful ways in the consultation 
process at the ECE facility, once you have worked with an ECE facility they will call back, and this takes tremendous 
energy to get a project up and running. 


The future of ECMH project will include consultations services for young children through the age of 5 years, additional 
training for ECE staff on MH and family issues, expanded education of legislators on the importance of social 
emotional development in young children, directed outreach about ECMH to families especially in rural counties, 
develop a system of continuing education for ECE staff focusing on social emotional development, increase 
professional development opportunities and support for ECMH staff including regularly scheduled reflective 
supervision, and expansion of consultant role to include systems building and possibly play therapy or 
parent/practitioner child interaction therapy while more professionals receive training in ECMH. 


Challenges and barriers for the coming year include getting the new regions on board, having new region staff learning 
the ropes while doing consultation and systems building simultaneously, being asked to fill the need in 3, 4, and 5 year 
olds, and having the Regional Key Directors provide the necessary supports to the ECMH staff in their region 


Additional supports needed by ECMH director and staff are support and acceptance from Regional Key, state offices, and 
each other (mentoring), identification of the specific skill set needed for the consultant/coordinator job, development of a 
statewide ECMH focus now that all regions are involved, and embed ECMH principles and staff in all relevant state offices to 
ensure a culture of ECMH to support all young children. 
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Appendix B 

Summary of Outreach and Systems Building Activities 


ECMH staff used a variety of outreach techniques to increase understanding, facilitate skill 
development, and create linkages with stakeholders. Stakeholders were from early care and 
education, early intervention, intermediate units, behavioral health, and health and human service 
agencies. Staff attended 144 meetings, conducted 68 trainings and professional development 
opportunities, and gave 61 presentations. Topics for trainings and professional development 
included challenging behaviors, social emotional development, inclusion, interacting with infants, 
and attachment. ECMH staff helped organize the Infant Toddler Mental Health Symposium and 
presented at conferences such as the Infant Toddler Mental Health Conference and Lebanon 
County Early Years. They also gave presentations to a variety of local and state groups that 
included Beginning with Books, Stars Technical Assistance, Local Interagency Coordinating 
Councils, Community Engagement Groups, and visiting professionals from the United Kingdom. 
In addition to formal outreach activities, ECMH staff worked hard to build relationships in 
informal ways such as phone calls, emails, dissemination and sharing of materials, and linking staff 
from different agencies and organizations. 


Listing of Outreach Activities in the Northwest, South Central, and Southwest 
Regional Keys 

144 Meetings (25%) took place with various agencies and organizations including Mind in 
the Making, Zero to Three, Early Childhood Mental Health Advisory Committee, Local 
Interagency Coordinating Council, Alliance for Infants and Toddlers, PA Regional Keys, PA 
Department of Health, Capital Area Early Childhood Training Institute, STARS Managers, 
Basic Steps, Early Childhood Mental Health Consultation, Early Intervention, Director of 
Comprehensive Autism Evaluation, Child and Adolescent Service System Provider, Leadership 
Council of the PA Child Care Association, PA Office of Child Development and Early Learning, 
Perinatal Partnership Stakeholders’ Task Force, Children’s Services Task Force, County 
Community Engagement Groups, County Director’s Roundtables, Court Teams, Office of 
Children, Youth, and Families, Butler County Integrated Access Unit, Crawford County Safe 
Kids, and Infant Toddler Mental Health. 

228 Building/maintaining relationships (39%) activities occurred with Child and 
Adolescent Service System Provider, Early Childhood Mental Health Advisory Committee, 

Infant and Early Childhood Mental Health coalition, Preventing Child Abuse and Neglect, 
Pittsburgh Public School teen parenting program, Women Infant and Children, Allegheny 
County System of Care, Chatham University, World Association of Infant Mental Health, PA 
Department of Public Welfare, County Mental Health and Mental Retardation, Open Forum 
for Practitioners, Capital Area Early Childhood Training Institute, Born to Read, Directors 
Roundtables, Keystone Stars, PA Keys Professional Development, Early Care and Education 
practitioners, Intermediate Unit, North East Alliance, Family Foundations, Early Intervention, 
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Early Head Start, Mathilda Theiss Center, PA Department of Health, professional development 
instructors, Child Care Information Services, PA Training and Technical Assistance Network, 
Child Development Associates programs, and local pediatricians. 

68 trainings/Professional Development (12%) were conducted including importance of 
social emotional development on school readiness, Ages and Stages Questionnaires: Social 
Emotional, inclusion, an overview of early childhood mental health, observation in infan t/toddler 
classrooms, toddler workshop, attachment, challenging behaviors, helping children manage anger, 
interacting and responding to infants, looking through different eyes at challenging behaviors, 
communicating with young children, dealing with biting, behavior management, healthy social and 
emotional development of young children, infant toddler mental health, supporting children’s 
social and emotional development, intervention on everyday settings, building relationships with 
infants and toddlers, responding to challenging behaviors in infants & toddlers, setting limits with 
children from birth to 5, and infan t/toddler issues. 

61 presentations (11%) were made to the following groups Beginning with Books, Starting 
Early Together team, Local Interagency Coordinating Council, Project Early Childhood Higher 
Education Opportunity, Greene County Perinatal Health Committee, Social Workers 
Professional Organization, United for Children, Pinnacle Health Infant Development Program, 
PA Child Care Association, STARS Technical Assistance, Lebanon County Early Years 
conference, Dr. Fienne and United Kingdom visiting professionals, United Way, Early 
Intervention, PA Training and Technical Assistance Network, County Community Engagement 
Groups, County Children and Youth staff, Northwest Regional Advisory, County home-based 
practitioners, and Crawford Safe Kids Coalition. 

53 resource development (9%) activities were completed including developed regional 
resource libraries, identified curricula (Promoting First Relationships, Mind in the Making, 
Parenting Counts), explored Maryland’s initiatives, compiled targeted resources for different 
stakeholders, utilized findings from Infant Toddler Build Task Force and Governors Commission 
for Children’s Mental Health, reviewed assessment tools, created handouts on how to make a 
referral and what is infant mental health, acquired counties’ community resource guides, developed 
resource list for counties, and installed “AutoLibrarian” software. 

24 trainings (4%) were attended by ECMH staff. They included the Zero to Three 
Conference, Transition Toolkit, Infant and Early Childhood Mental Health Summit, Pittsburgh 
Association for the Education of Young Children Conference, Reflective Supervision, Mind in the 
Making Facilitator Training, Perinatal Depression Work Group, Supporting Adult Learners 
Implementing Routines Based Early Intervention, What Do You Do With The Mad That You 
Feel, Mind in the Making Professional Development, the Ounce Scale, Moving to a Collaborative 
Consultative Model for Service Delivery in Early Intervention, Positive Beginnings Modules for 
Challenging Behaviors, Building Relationships with Families, and The Young Child with Special 
Needs Conference. 


Outreach Activity Hours : 952.75 hours total: 475.5 (50%) hours of direct contact, 

352.8 (37%) hours of travel, 124.5 (13%) hours of communication 
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Appendix C 

Summary of Consultation Records 


Description of Children and Programs Using Consultation Services 

The ECMH Project provided consultation services to early learning facilities seeking help 
for specific children in their programs. From September 20, 2006 to June 30, 2008, 133 programs 
requested assistance with children for whom they had social emotional or behavioral concerns. 
Consultation was provided for 226 children between the ages of 6 weeks and 5% years. At the 
start of services, 82% of children were toddlers (12-35 months), 11% were preschoolers (>. 36 
months), and 6% were infants (<12 months). Consultation was most frequently initiated for 
children between 29 and 35 months (29% of cases); and the mean age of children at the start of 
services was 28 months. 

Practitioners evaluated children’s social emotional skills on the Ages and Stages 
Questionnaires: Social Emotional (ASQj^ SE) at the start of consultation. One hundred and 
thirty-eight ASQ^ SE assessments were completed for children during the pilot. Children’s skill 
levels ranged considerably, but “on average’ children’s scores reflected social emotional difficulties 6 
Consultants supported practitioners for longer periods of time when children had less well- 
developed social emotional skills and/ or more challenging behaviors. The finding is based on a 
very moderate relationship (r=.237, p<.01) between children’s ASQj^ SE scores and the length of 
consultation services. 


Table 5. Practitioner Ratings of Children’s Skills on the ASQ: SE and 
Consultant Ratings of Caregiver-Child Interactions on the CIS 


Mean Score 

Range of Scores 

Standard Deviation 

Number of 
Assessments 


Children’s social 
emotional skills 

ASQ:SE Score 

95 

15-240 

51 

138 


Quality of teacher- 
child interactions 

CIS Score 

3.4 

1 -4 
.5 

126 


6 Mean “average” score = 95, median “50 th percentile” score = 85, mode or “most frequent” score = 80, and standard deviation 
“averaged score variation” = 51. 
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ECMH consultants assessed the quality of interaction between children for whom services 
were requested and their teachers or family day care providers. The Caregiver Interaction Scale 
(CIS) was used for this purpose. The quality of interactions between caregivers and children 
ranged from low ratings of 1 to high ratings of 4. The “average” quality of interactions or mean 
CIS score was 3.4. No significant relationship was found between the quality of teacher child 
interaction and length of consultation services. As noted earlier, 133 programs requested assistance 
with children for whom they had social emotional or behavioral concerns. Services were provided 
primarily to center-based programs located in suburban (46%) and urban (37%) settings. Eighty- 
nine percent of the early care and education settings were centers, 8% were family day cares, 2 % 
were nursery schools, and 1% were group homes. Most programs had either no staff (31%) or one 
staff with Associates degrees or higher (50%) and one to two staff with early childhood 
professional development training in the last three years (71%). 


Primary Reasons for ECMH Consultation Services 

Programs requested assistance for a variety of reasons. Consultants classified reasons into 
one of five categories, (1) attachment or interaction issues, (2) self-regulatory issues (over- activity, 
under-activity, irritability, etc.), (3) communication issues, (4) acting out/aggressive behaviors, and 
(5) other. Help was most frequently sought for children experiencing difficulties with self- 
regulation (29%), aggression/acting out (29%), and attachment/interaction (27%). A summary 
of findings is provided on the next page in Table 6. Programs varied slightly across regional keys, 
geographic settings, and programs. 

• Practitioners most frequently sought help for, 

aggression and acting out behaviors in the Northwest Regional Key, 
attachment/interaction and self-regulation issues in the South Central Regional Key, 
self-regulation issues in the Southwest Regional Key. 

The percentage of requests from practitioners in the South Central and Southwest Regional 
Keys was fairly similar across these areas (27-36%). Assistance with aggressive behaviors was 
requested most frequently by practitioners in the Northwest Key (55% of the time). 


• Programs in the following settings most frequently sought help for, 

attachment/interaction issues and aggression and acting out behaviors in urban settings, 
attachment/interaction issues, self-regulation issues, and aggression and acting out 
behaviors in suburban settings, and 
self-regulation issues in rural settings. 

Overall, the percentage of child concerns appeared somewhat similar (27-34%) across each 
setting. 

• Help was most frequently sought by STAR 1 programs and least frequently by ‘traditional’ 
STAR 4 programs. A surprising finding was that STAR 4 programs that reached this 
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designation through NAEYC accreditation were much more likely to request consultation 
services than those that reached a STAR 4 designation through Pennsylvania’s Keystone 
STARS Performance Standards (15 %vs. 2%). 

• With the exception of STAR 4, programs tended to seek help for children with 
attachment/interaction issues, self-regulation issues and aggression and acting out 
behaviors. STAR 4 programs only requested ECMH services on four occasions; assistance 
was asked for around attachment and interaction. 

Table 6. Primary Reason Programs Requested ECMH Consultation Services for Children 

Primary Reason for Request of ECMH Services * 



Percentages based 
on 

Attachment / 

Interaction 

Issues 

Self-Regulation Communication 
Issues Issues 

Acting Out/ 
Aggression 

Other 

Regional Keys 







NW, SC, & SW 

223 cases 

27% 

29% 

7% 

29% 

8% 

Northwest 

40 cases 

25% 

13% 

5% 

55% 

3% 

South Central 

1 1 2 cases 

29% 

31% 

9% 

24% 

7% 

Southwest 

61 cases 

25% 

36% 

5% 

21% 

13% 

Programs Settings 

221 cases 






Urban 

65 cases 

29% 

22% 

8% 

34% 

8% 

Suburban 

1 1 6 cases 

27% 

32% 

6% 

28% 

7% 

Rural 

40 cases 

23% 

33% 

10% 

25% 

10% 

STARS Level of Early 

1 96 cases 






Learning Programs 







Starting with STARS 

27 cases 

22% 

33% 

3% 

33% 

7% 

STARS Level 1 

79 cases 

20% 

35% 

10% 

30% 

4% 

STARS Level 2 

31 cases 

29% 

19% 

10% 

29% 

13% 

STARS Level 3 

25 cases 

28% 

28% 

4% 

28% 

12% 

STARS Level 4/ 

30 cases 

20% 

30% 

3% 

33% 

13% 

accreditation 







STARS Level 4 

4 cases 

100% 

0% 

0% 

0% 

0% 


*226 children received services 
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Referrals made by ECMH Consultants 


ECMH consultants made referrals to other services for 58% of their cases (n =130). These 
services included early intervention, child and adult mental health, STARS TA, and other services 
such as pediatricians, audiologists, and other early learning programs. The majority of these 
referrals were to early intervention (48%) and STARS TA (16%) and in a notable number of cases 
(23%, n= 30) referrals were made to multiple support services. A summary of consultants’ referrals 
is provided below. 


El 

52% 

El & Child Mental Health 

5% 

STARS TA 

16% 

Child Mental Health 

2% 

El 8c Other (docs, ECE’s) 

8% 

El 8c Adult Mental Health 

1.5% 

El 8c STARS TA 

7% 

El, STARS TA, 8c Other 

1.5% 

Other (docs, ECE’s, etc.) 

5% 

El, Child 8c Adult Mental Health 

<1% 


Most cases were discharged from ECMH services because consultation goals were achieved 
or referrals were made to another service such as early intervention or STARS TA. This finding 
was consistent across Regional Keys, geographic locations, and STARS designations. A summary 
of these findings is provided in Table 7. 


Table 1. Reason for Discharge from ECMH Consultation Services 



Met Goals 

Referred to 

Family 

Provider 

Family 

Percentages 



Another 

Service 

Choice 

Choice 

Moved 

based on 

Regional Keys 
NW, SC, & SW 

39 % 

30% 

15% 

10% 

7% 

1 92 cases 

Northwest 

61 % 

18% 

15% 

3% 

3% 

33 cases 

South Central 

26% 

41 % 

10% 

16% 

7% 

1 07 cases 

Southwest 

50 % 

14% 

23% 

4% 

10% 

52 cases 

Geographic Location 
of Early Learning 
Programs 
Overall 

39 % 

30% 

15% 

10% 

7% 

191 cases 

Urban 

33 % 

35 % 

18% 

8% 

6% 

51 cases 

Suburban 

37 % 

27% 

17% 

12% 

7% 

1 05 cases 

Rural 

51 % 

31% 

3% 

6% 

9% 

35 cases 

STARS Level of Early 
Learning Programs 






170 cases 

Starting with STARS 

33% 

42 % 

13% 

8% 

4% 

24 cases 

STARS Level 1 

34 % 

34 % 

12% 

16% 

12% 

67 cases 

STARS Level 2 

36 % 

29 % 

18% 

11% 

7% 

28 cases 

STARS Level 3 

52 % 

13% 

26% 

4% 

4% 

23 cases 

STARS Level 4/ 
accreditation 

42 % 

33% 

17% 

4% 

4% 

24 cases 

STARS Level 4 

0% 

50 % 

25% 

25% 

0% 

4 cases 
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Methodology 


Consultants gathered information from early learning programs once requests were made for 
help with individual children. In addition to the nature of the request/child concerns, this 
information included program demographics and consultation activities. Program demographics 
consisted of contact information, staffing, education and recent professional development trainings, 
STARS level, and geographic region. Consultation activity records included initial observations, 
child and classroom screening scores, completion of service plans, referrals for other services, 
discharge date and reason, and additional notes as needed. On occasion, programs sought services for 
the same children a second time. In these instances, case data was gathered again. Consultants 
maintained records for 234 cases between September 2006 and June 2008. Eight records appear to 
be duplicates or errors and as such were excluded from all analyses. Updated case data was submitted 
by consultants to the ECMH Project Manager on a monthly basis. 
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Appendix D 

Summary of Practitioners’ 
Perspectives of ECMH Consultation Services 


The project measured the impact of the ECMH consultation through Program Feedback 
Surveys collected from early care and education (ECE) practitioners. Results were very positive 
and respondents gave high praise for the project. Respondents clearly valued working with ECMH 
consultants and saw on-site assistance as the most valued and helpful aspect of the project. 
Respondents noted positive changes in the perceptions, behaviors, and skills of staff as well as 
positive, though gradual, changes in the children’s behaviors. These changes were the direct result 
of programs’ interactions with ECMH consultants. Programs reported that while consultation 
facilitated the development of new skills in many providers, it primarily helped practitioners meet 
the socio-emotional needs of children for whom support was requested. All respondents stated 
that they would recommend ECMH services to others (n=50, 100%). The following comments 
reflect their enthusiasm for the project: 

“I would recommend this program to anyone. If it was not for [the ECMH 
consultant ], this child would not be allowed to stay enrolled in our program. ” 

“[The ECMH Program] has been a Godsend to our center. We have received 
information and support for children in need. [The ECMH consultant] is 
terrific to work with. ” 

“It has been the most valuable resource I have experienced in my 10 years 
directing this center. ” 


Table 8. Helpfulness of ECMH Consultation Services 


Service 


On-site assistance 

Support with child’s 
family 

Referrals for child to 
other services 

Resource materials 


How helpful was this service? 


Not At All 
Helpful 

Somewhat 

Helpful 

Very 

Helpful 

Extremely 

Helpful 

Number of 
Respondents 
Using 
Service 


10% 

20% 

69% 

49 out of 50 

5% 

16% 

26% 

54% 

43 out of 50 

3% 

7% 

27% 

63% 

30 out of 48 


28% 

30% 

43% 

40 out of 50 


28% 

36% 

36% 

25 out of 50 


Referrals for 
professional 
development 
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Respondents were asked to describe changes they observed in staff and in children as a 
result of working with the ECMH consultant. The majority of respondents observed positive 
changes in staff perceptions (n=39, 80%) and staff behavior/practices (n=41, 87%). One 
respondent commented on changes in staff perceptions stating, “[Staff] now realize that she is a 
child with special needs that we need to deal with differently than our other children.” As a result 
of their interactions with ECMH consultants, respondents observed that staff became “less 
stressed,” “stopped yelling,” and “took more time with child(ren).” 

Moreover, respondents observed positive changes in the behavior of the children with 
whom the ECMH consultant assisted (n=31, 72%). While most of the respondents conceded that 
improvements were gradual and required ongoing work, they recognized and were grateful for 
behavior changes such as fewer and shorter tantrums and less crying. One respondent commented 
that s/he felt the child “feels more accepted now and that we understand her better than before.” 

Despite working with an ECMH consultant, 10 children (20%) left the child care program 
they had been attending. However, it does not appear that the children left the program because 
of behavioral issues, but rather because of logistical issues such as the child’s family moving. 

Overall, most of the respondents described the quality of the assistance they received from 
the ECMH consultant as excellent (n=35, 70%) or very good (n=ll, 22%). Most of the 
respondents described the ECMH consultant’s assistance as extremely useful (n=33, 66%) or very 
useful (n=ll, 22%). However, six respondents (12%) felt the ECMH consultant’s assistance was 
only somewhat helpful. In one case, the respondent was hoping for someone to work directly with 
the child, rather than providing support for the provider and parent. In other cases, the 
respondents described the child as “very determined” or “would have liked more suggestions on 
different behavioral techniques” but conceded that “some of the information was useful.” 

Nearly all of the respondents felt that they will be able to use the information and strategies 
they gained from ECMH consultants to some extent (n=12, 25%) or quite a bit (n=34, 71%) with 
other children in their program. This indicates that the staff will apply the information and 
strategies they learned to other children in their program, thus expanding the impact of the 
consultation. 

ECMH consultants offered a variety of assistance to providers, including on-site assistance, 
support with the child’s family, referrals for the child to other services, resource materials, and 
professional development referrals for ECE staff. Respondents rated these services in terms of 
their usefulness and these ratings are listed below. Overall, the provision of on-site assistance 
appears to be the service that was most frequently taken advantage of by programs and was also the 
service that was most helpful to providers. 

One of the goals of the ECMH Project is to increase providers’ skills in understanding how 
children’s history effects their current behavior, understanding family’s current situations, making 
referrals for needed services for children, meeting the socio-emotional needs of children 
experiencing difficulties, and meeting the socio-emotional needs of other children in the provider 
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setting. Respondents were asked to rate how their skill level changed in each of these areas since 
working with the ECMH consultant; these ratings are listed in the table below. Overall, the 
majority of respondents felt that their work with the ECMH consultant somewhat or greatly 
improved their skill level in every area. This was especially true in the area of meeting the socio- 
emotional needs of the specific child for whom assistance was requested. In this case, 85% of the 
respondents felt their interaction with the ECMH consultant somewhat or greatly improved their 
skills to meet the specific child’s socio-emotional needs. 

Table 9. Practitioners’ Skills after Working with ECMH Consultants 



How did the work with the 

Number of 


consultant affect your skill level? 

respondents 



Somewh 


engaged in 

Skill 

No 

at or 

Already 

building 

Change 

Greatly 

Improved 

Proficient 

these skills? 


Understanding child’s history and current 
behavior 

14% 

71% 

14% 

42 out of 50 

Understanding family situation 

14% 

64% 

21% 

42 out of 50 

Making referrals for child 

6% 

73% 

21% 

34 out of 50 

Meeting socio-emotional needs of this child 

6% 

80% 

14% 

32 out of 50 

Meeting socio-emotional needs of other 
children 

4% 

79% 

17% 

48 out of 50 


Methodoloc 


Beginning in June 2007, practitioners were asked to complete feedback surveys at the end 
of consultation 7 . Survey questions focused on the quality and usefulness of assistance received 
from ECMH consultants. Fifty individuals from provider settings in the South Central, 
Northwestern, and Southwestern regions completed surveys. Most surveys were completed by 
facility directors (n=29, 58%), while other respondents included teachers (n=ll, 22%), assistant 
teachers (n=l, 2%), child care providers (n=7, 14%), and other staff (n=2, 4%). Between June 1, 
2007 and June 30, 2008 approximately 145 children were discharged from services 8 . However, the 
number of feedback surveys sent by consultants to providers is unclear. As such, a survey response 
rate was not calculated. 


7 Program Feedback Surveys were developed at the end of Pilot Year One. 

s Discharge dates missing for 8 cases. 
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Appendix E 

Summary of Stakeholders’ Perceptions of the ECMH 
Project’s Impact on the Northwest, South Central, and 
Southwest Regional Keys 


Based on feedback from the 85 key stakeholders, ECMH Project was successful in 
increasing awareness and access. The project made great strides in increasing awareness around 
infant toddler mental health including typical and atypical development, the need to address these 
issues in young children, and the available services and supports for families and practitioners with 
young children. The ECMH Project also increased access to educational materials, referrals, and 
mental health and medical services including on-site services for young children. However, more 
work needs to be done. Important issues to focus on, as rated by respondents, included training for 
staff and parents on child development and infant toddler mental health, support for quality early 
care and education programming, and better linkages between agencies that support and serve 
young children’s mental health needs. Respondents identified three issues that need to be addressed 
first, the need for more on-site resources to support young children, for stronger linkages between 
systems that serve young children and more assistance for timely and appropriate referrals for 
young children with mental health issues. Respondents recognized a need for more ECMH staff 
to do the work and more marketing to “get the word out” about the ECMH Project. Lastly, 
respondents requested the following resources to better support young children in their own work, 
additional educational materials for self and families, more opportunities to network with other 
professionals around the issue of infant toddler mental health, and more training opportunities 
focusing on infant toddler mental health and child development. 

Respondents rated the effect of the ECMH Project on both awareness and access in seven 
and nine areas, respectively. Respondents cited the ECMH Project as increasing awareness in 5 out 
of 7 (71%) areas including awareness of services and supports, patterns of typical and atypical 
development, how to identify mental health issues in young children, strategies to support social 
emotional development, and the need to address young children’s mental health concerns. However, 
about half the respondents did not see any changes in awareness in the areas of “how to work through 
the barriers and constraints of the current infant toddler mental health system” and “mental health 
agencies’ understanding parents’ experiences with their policies, procedures, and services.” See Table 
10 for complete list of stakeholders’ ratings. 
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Table 10. Awareness of Infant Toddler Mental Health Issues 



Decreased 

Awareness 

No change 
in 

Awareness 

Increased 

Awareness 

Don’t 

know 

Response 

Count 

Services and supports for 
infants/toddlers with mental 
health issues 

0.0% (0) 

32.9% (26) 

49 . 4 % ( 39 ) 

17.7% (14) 

79 

How early learning 
practitioners can identify 
mental health issues in young 
children 

0.0% (0) 

30.4% (24) 

46 . 8 % ( 37 ) 

22.8% (18) 

79 

Patterns of typical and 
atypical child development 

0.0% (0) 

35.4% (28) 

46 . 8 % ( 37 ) 

17.7% (14) 

79 

Developmentally appropriate 
strategies that support young 
children's social emotional 
development 

0.0% (0) 

32.9% (26) 

51 . 9 % ( 41 ) 

15.2% (12) 

79 

Importance of addressing 
young children's mental 
health concerns earlier rather 
than later 

0.0% (0) 

21.5% (17) 

65 . 8 % ( 52 ) 

12.7% (10) 

79 

How to work through the 
barriers and constraints of the 
current infant toddler mental 
health system 

1.3% (1) 

50 . 0 % ( 39 ) 

30.8% (24) 

17.9% (14) 

78 

Mental health agencies’ 
understanding of parents’ 
experiences with their 
policies, procedures, and 
services 

0.0% (0) 

51 . 3 % ( 40 ) 

24.4% (19) 

24.4% (19) 

78 


Respondents reported the project had affected slightly fewer areas in terms of access. Only 
two thirds (6 out of 9) of the areas pertaining to access were rating as having increased since the 
beginning of the ECMH project in July 2006. Areas identified by the respondents with increased 
access included Early Intervention and medical services, mental health services within the early 
learning facility, educational materials about infant toddler mental health, professional 
development focusing on infant toddler mental health, referral assistance, and support for 
collaborations between agencies serving young children and their families. The three areas 
respondents reported “no change” in access were “age appropriate mental health services,” “infant 
toddler mental health assessments,” and “assistance with referrals for infant toddler mental health 
services.” See Table 11 for a complete list of stakeholders’ ratings. 
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Table 11. Access to Infant Toddler Mental Health Resources 



Decreased 

Access 

No change 
in Access 

Increased 

Access 

Don’t 

know 

Response 

Count 

Age appropriate mental health 
services 

2.7% (2) 

47 . 9 % ( 35 ) 

28.8% (21) 

20.5% (15) 

73 

Mental health services provided 
within the early learning facility 

2.7% (2) 

35.6% (26) 

46 . 6 % ( 34 ) 

15.1% (11) 

73 

Other support services such as 
Early Intervention or medical 
services 

2.7% (2) 

28.8% (21) 

56 . 2 % ( 41 ) 

12.3% (9) 

73 

Educational materials about infant 
toddler mental health 

1.4% (1) 

31 .5% (23) 

54 . 8 % ( 40 ) 

12.3% (9) 

73 

Infant toddler mental health 
assessments 

1.4% (1) 

41 . 1 % ( 30 ) 

35.6% (26) 

21.9% (16) 

73 

Professional development 
opportunities in the area of infant 
toddler mental health 

1.4% (1) 

35.6% (26) 

45 . 2 % ( 33 ) 

17.8% (13) 

73 

Assistance with referrals for infant 
toddler developmental services 

2.7% (2) 

35.6% (26) 

43 . 8 % ( 32 ) 

17.8% (13) 

73 

Assistance with referrals for infant 
toddler mental health services 

2.7% (2) 

39 . 7 % ( 29 ) 

38.4% (28) 

19.2% (14) 

73 

Support for collaborations between 
agencies serving young children 
and their families 

2.7% (2) 

32.9% (24) 

49 . 3 % ( 36 ) 

15.1% (11) 

73 


In addition to measuring respondents’ perception of increases in awareness and access, the 
web survey asked respondents to rate the importance of addressing issues in the areas of the 
childcare system, policy system, and relationships across systems. Issues were rated in terms of the 
importance of being addressed by the ECMH Project and the importance of being addressed 
overall. Ratings were based on a four point scale with 1 being “not at all important,” 2 being 
“slightly important,” 3 being “important,” and 4 being “very important.” Respondents rated ten 
issues in the childcare system with the resulting average ratings ranging from 3.27 to 3.59, five 
issues in the policy system with average ratings ranging from 2.9 to 3.59, and ten issues in 
relationships across systems with average ratings ranging from 2.87 to 3.48. 
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Table 12. Importance for Addressing Infant Toddler Issues 



Not at all 
Important 

Slightly 

Important 

Important 

Very 

Important 

Average 

Rating 

Childcare System 

Increasing parental knowledge of 
child development and infant 
toddler mental health issues 

3.0% (2) 

1.5% (1) 

28.8% (19) 

66.7% (44) 

3.59 

Receiving on-site assistance 
while waiting for formal 
supports/services to begin 

0.0% (0) 

4.8% (3) 

38.1% (24) 

57.1% (36) 

3.52 

Receiving on-site resources 

0.0% (0) 

1.6% (1) 

45.2% (28) 

53.2% (33) 

3.52 

Policy System 

Promoting linkages between 
systems that serve young 
children 

0.0% (0) 

1.4% (1) 

38.6% (27) 

60.0% (42) 

3.59 

Supporting quality ECE 
programming 

0.0% (0) 

2.9% (2) 

44.9% (31) 

52.2% (36) 

3.49 

Increasing awareness of the 
depth and breath of infant toddler 
mental health issues 

0.0% (0) 

8.8% (6) 

47.1% (32) 

44.1% (30) 

3.35 

Relationships Across Systems 

Facilitating timely and appropriate 
referrals 

0.0% (0) 

4.3% (3) 

43.5% (30) 

52.2% (36) 

3.48 

Providing training for staff and 
parents around the issues of early 
childhood mental health 

0.0% (0) 

5.7% (4) 

48.6% (34) 

45.7% (32) 

3.41 

Supporting linkages between 
systems serving young children 

0.0% (0) 

0.0% (0) 

58.8% (40) 

41.2% (28) 

3.40 


The most important issue to be addressed by the ECMH Project in each of these areas 
was providing on-site resources, promoting linkages between child serving systems and 
facilitating timely and appropriate referrals . 9 Respondents’ ratings for the top three childcare, 
policy, and cross -systems issues are summarized in the table above. Child care system issues that 
respondents identified as being important to address were “increasing parental knowledge of child 
development and infant toddler mental health issues,” “receiving on-site resources,” and “receiving 


9 These areas are bolded on the table. 
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onsite assistance while waiting for formal supports/services to begin.” The top three issues in the 
policy system were “promoting linkages between systems that serve young children,” “supporting 
quality ECE programming,” and “increasing awareness of the depth and breadth of infant toddler 
mental health issues.” The top three issues in the area of relationships across systems were 
“facilitating timely and appropriate referrals,” “supporting linkages between systems serving young 
children,” and “providing training for staff and parents around the issues of early childhood mental 
health.” See Table 12 for summary of ratings for top issues. 

When asked to comment on what they would change about the ECMH Project, forty 
respondents offered a variety of suggestions including expanding the age of children served to 
include preschoolers and offering training for parents. However, the two most common 
suggestions were increase marketing efforts and increase ECMH staff. A quarter of the 
respondents suggesting more marketing efforts were needed to raise awareness about the project 
especially in rural areas. One respondent suggested making a commercial about ECMH Project. 
The other popular suggestion given by 20% of the respondents was the need for more staff to 
expand the reach of the ECMH Project. 

Lastly when asked to identify which of the eight listed resources their agency needed to 
better address infan t/toddler mental health issues, 67% of the respondents chose “educational 
materials for self or to share with parents.” Respondents also wanted “opportunities to network 
with practitioners or professionals who are concerned about infant toddler mental health issues.” 
This was identified by 62% of the respondents and slightly more than half of the respondents 
(54%) asked for “training on infant toddler mental health or child development.” 


Methodology 

Stakeholders were surveyed in the spring of 2008 to measure perceptions of the project’s 
impact on changes in awareness of early childhood mental health issues and access to services. In 
addition, they were asked to identify key childcare, policy, and systems issues and they were asked 
to provide general feedback about the ECMH Project. The areas addressed in this survey were 
based on results from the initial stakeholders’ survey, conducted in the fall of 2006 10 . 

ECMH staff identified stakeholders from the Northwest, South Central, and Southwest 
Regional Keys. Based on this list, 300 stakeholders were contacted via e-mail to participate in a 
web-based survey. Eighty-five respondents completed the survey 11 . Stakeholders responded from 
all counties across the three regions. The largest percentage of respondents were from Erie County 
(n=ll, 41%) in the Northwest, Allegheny County (n=15, 94%) in the Southwest, and Lancaster 
County (n=18, 67%) in the South Central key. Seventy-one percent of respondents provided 
demographic information, 66% reported education as the subject of their professional training with 
half receiving their Bachelors’ degrees and a quarter their Master’s degree. There was a range of 


10 These results were summarized in a report issued in April 2007, referenced in the first year report, and are Table 4 in Appendix 
A of this report. 

1 1 This number corresponds to a response rate of 28%. 
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experience working with children from one to forty years with an average of twenty years. Slightly 
over half of the respondents (51%) work in early learning facilities and about 13% work in Early 
Intervention. Over three-quarters (78%) hold an administrator or owner position in their 
workplace. The respondents also reported various levels of contact with the ECMH Project 
including 35% attended a training or presentation on the project, 28% knew about the project but 
had no direct involvement, 27% received services from ECMH staff, 18% were interviewed by 
DARE for the first stakeholders report, and only 17% had no knowledge of ECMH Project prior 
to completing the survey. 
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